Method and system to characterize video background changes as abandoned or removed objects

ABSTRACT

A method and system for analyzing video data in a security system. An analysis compares a current frame to a background model. The analysis system compares the background model to the current frame to identify changed pixel patches. The analysis system uses morphological image processing to generate masks based on the changed pixel patches. Next, the analysis system applies the masks to the background model and the current frames to determine whether the changed pixel patches are characteristic of abandoned or removed objects within the video data.

RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.14/962,533, filed on Dec. 8, 2015, which is a Division of U.S.application Ser. No. 13/783,625, filed on Mar. 4, 2013, both of whichare incorporated herein by reference in their entirety.

BACKGROUND OF THE INVENTION

Video security systems include security cameras and are often used forsurveillance and video data analysis. These security systems are used tomonitor buildings, lobbies, entries/exits, and secure areas within thebuildings, to list a few examples. Additionally, the security systemsare also used to monitor public areas such as garages, parking lots,building exteriors, and other areas in or around the buildings.Generally, these security systems are implemented to monitor usage butalso to identify illegal activity such as theft or trespassing, to lista few examples.

Modern video security systems have the capability to analyze the videodata captured by the security cameras. Typically, these systems are ableto track individuals and possibly notify security personnel ifunauthorized persons are entering (or exiting) a restricted area, forexample. Additionally, the security systems also monitor objects withinscenes. For example, abandoned objects (e.g., unattended backpacks orpackages) should be identified in airport terminals, stadiums, orconvention centers, for example. On the other hand, security personnelshould be notified if objects are removed from a museum or ifmerchandise is removed from a retail establishment.

Abandoned or removed objects in scenes are identified using an analysissystem. It analyzes the video data from the security cameras to generatea background model. The background model may be, for example but notlimited to, a single video frame occurring prior to the backgroundchange, or an analysis of frames over time. Then current frames of videodata from those security cameras are compared against the backgroundmodel to identify changed pixel patches.

In one example, changed pixel patches were identified and used to mask achanged area in the current frames of video data relative to thebackground model. These changed pixel patches were used by the analysissystem to conclude whether some part of the scene, such as an object inthe scene, had changed. Next, the analysis system detected the number ofedges in the changed area of the current frame and the background model.If there were more edges in the current frame than the background model,then the analysis system concluded that an object had been abandoned inthe scene. In contrast, if there were fewer edges detected in thecurrent frame than in the background model, then the analysis systemconcluded that an object had been removed from the scene.

In another example, the analysis system measured similarities betweencontent inside and outside of the changed pixel patches in the currentframe of video data. If the similarities were above a predefinedthreshold, then the analysis system concluded the object had beenremoved from the scene. Conversely, if the similarities of the contentwere below the predefined threshold, then the analysis system concludedthe object had been abandoned in the scene.

In another example, the analysis system analyzed contours around thechanged pixel patches in the current frame of video data. Then theanalysis system compared the contours of the changed pixel patches tothe edges detected in masked areas of the current frame. For each pixelpatch or group of patches corresponding to an object, if similaritiesbetween the detected edges and the contours exceeded a predefinedthreshold, then the analysis system concluded that the object had beenabandoned in the scene. If the similarities between the detected edgesand the contours did not exceed the predefined threshold, then theanalysis system concluded that the object had been removed from thescene.

SUMMARY OF THE INVENTION

These previous analysis systems were often unable to reliablydistinguish between abandoned or removed objects in scenes. For example,the analysis systems often made mistakes in concluding whether an objecthad been abandoned or removed when the background had more texture thanthe object (e.g., a plate on a Persian rug). Likewise, the analysissystems also made mistakes comparing the similarity of content insideand outside of the changed pixel patches when contours of the changedpixel patches did not accurately match contours of the object (e.g.,because of poor segmentation). Additionally, the analysis system oftenmade mistakes comparing contours for the changed pixel patches to theedges detected in the masked area of the current image when thebackground has similar features as foreground objects. For example, astack of similar items (e.g. newspapers or jeans) and then the top itemis removed.

In the present solution, the analysis system performs an imagesubtraction operation between current frames and a background model toidentify changed pixel patches. The analysis system then analyzes thechanged pixel patches and detects edges in the background model that lienear a contour of the changed pixel patch and measures a strength of thedetected edges. Next, the analysis system detects edges in the currentframe that lie near the contour of the changed pixel patches andmeasures a strength of those detected edges. If the strength of thedetected edges of the background model is greater than the strength ofthe detected edges of the current frame by a threshold, then the changedpixel patches are characterized as removed objects. If the strength ofthe edges of the background model is less than the strength of the edgesof the current frame edges, then the changed pixel patches arecharacterized as abandoned objects.

Additionally, the present solution is beneficial because it does notrequire accurate segmentation, it does not assume relative edge densityof foreground versus background image patches, and it does not assume anabsolute image edge density. Therefore, the present solution is able toovercome many of the problems associated with previous analysis systems.

In general, according to one aspect, the invention features a method foranalyzing video data in a security system. The method includes comparinga background model to at least one frame of the video data to identifyat least one patch of changed pixels between the background model andthe at least one frame. The method further includes generating masksbased on the at least one patch of changed pixels that define contoursaround the at least one patch of changed pixels and applying the masksto the background model and the at least one frame to determine whetherthe at least one patch of changed pixels is characteristic of anabandoned object or a removed object.

In general, according to another aspect, the invention features a methodfor analyzing video data in a security system. The method includescomparing a background model to at least one frame of the video data toidentify at least one patch of changed pixels between the backgroundmodel and the at least one frame. The method further includes summinggradient magnitudes for edge pixels in the background model with respectto the at least one patch of changed pixels. Additionally, the methodincludes summing gradient magnitudes for edge pixels in the at least oneframe of video data with respect to the at least one patch of changedpixels and characterizing objects in the video data as abandoned orremoved based on a comparison of the sums of gradient magnitudes betweenthe background model and the at least one frame.

In general, according to another aspect, the invention features asecurity system that includes cameras to capture video data and anetwork video recorder to store the captured video data. Additionally,the security system includes an analysis system to compare a backgroundmodel to at least one frame of the video data to identify at least onepatch of changed pixels between the background model and the at leastone frames. The security system then generate masks based on the atleast one patch of changed pixels that define contours around the atleast one patch of changed pixels and applies the masks to thebackground model and the at least one frame to determine whether the atleast one patch of changed pixels is characteristic of abandoned objectsand/or removed objects.

In general, according to another aspect, the invention features asecurity system that includes cameras to capture video data and anetwork video recorder to store the captured video data. The securitysystem includes an analysis system that compares a background model toat least one frame of the video data to identify at least one patch ofchanged pixels between the background model and the at least one frame.The analysis system then sums gradient magnitudes for edge pixels in thebackground model with respect to the at least one patch of changedpixels and sums gradient magnitudes for edge pixels in the at least oneframe of video data with respect to the at least one patch of changedpixels. Next, the analysis system characterizes objects in the videodata as abandoned or removed based on a comparison of the sums ofgradient magnitudes between the background model and the at least oneframe.

The above and other features of the invention including various noveldetails of construction and combinations of parts, and other advantages,will now be more particularly described with reference to theaccompanying drawings and pointed out in the claims. It will beunderstood that the particular method and device embodying the inventionare shown by way of illustration and not as a limitation of theinvention. The principles and features of this invention may be employedin various and numerous embodiments without departing from the scope ofthe invention.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings, reference characters refer to the sameparts throughout the different views. The drawings are not necessarilyto scale; emphasis has instead been placed upon illustrating theprinciples of the invention. Of the drawings:

FIG. 1A is a block diagram showing a security system to which thepresent invention relates.

FIG. 1B is a block diagram showing a security system with anotherarchitecture to which the present invention also relates.

FIG. 1C is a block diagram showing a security system with still anotherarchitecture to which the present invention also relates.

FIG. 2 is a flowchart illustrating the steps performed by the analysissystem of the security system to analyze frames of video data capturedby security cameras.

FIG. 3 is a flow chart illustrating the steps performed by the analysissystem to characterize background objects as abandoned or removed.

FIG. 4 illustrates how the inner mask is modified by the imageprocessing steps.

FIG. 5 is an alternative embodiment of how the analysis systemcharacterizes background objects as abandoned or removed.

FIG. 6A illustrates a background model image.

FIG. 6B illustrates an object being abandoned.

FIG. 6C illustrates an object being removed.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention now will be described more fully hereinafter withreference to the accompanying drawings, in which illustrativeembodiments of the invention are shown. This invention may, however, beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the invention to those skilled in the art.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Further, the singular formsof the articles “a”, “an” and “the” are intended to include the pluralforms as well, unless expressly stated otherwise. It will be furtherunderstood that the terms: includes, comprises, including and/orcomprising, when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof. Further, it will be understood that when anelement, including component or subsystem, is referred to and/or shownas being connected or coupled to another element, it can be directlyconnected or coupled to the other element or intervening elements may bepresent.

FIGS. 1A, 1B, and 1C illustrate different embodiments of the videosecurity system 100, which includes security cameras 110, a networkvideo recorder 112, and a video data and metadata archive (or archive)114.

The security system 100 includes one or more video cameras 110, whichare connected to the network video recorder 112 via a network 104.Typically, the network 104 is a private network, such as a local areanetwork provided within the building. In other embodiments, the network104 includes a combination of private and/or public networks so that thevideo data from the security cameras 110 are able to be transmitted tothe network video recorder system 112 from remote locations.

The network video recorder 112 stores the video data and any metadata inthe archive 114. The metadata are data that describes the captured videodata. For example, the metadata may include a camera identificationnumber (or name), the locations of the camera in the building, itsInternet Protocol address (when implemented on a data network), movementof foreground objects, and/or events of interest, to list a fewexamples. Generally, the archive 114 includes both a video store 116that includes the raw video data from the cameras 110 and a metadatastore 118 for storing metadata associated with the captured video data.

The video data generated by the security cameras 110 are analyzed by ananalysis system 115. This analysis system 115 generates the metadatafrom the video data captured by the security cameras 110. The securitysystem 100 stores the metadata associated with the video data in themetadata store 118 so that security personnel 126 and/or users are ableto search through the video data for specific events in the archivedvideo data later. The security personnel search for objects beingabandoned in the scene or objects being removed from the scene, in twoexamples.

In a typical implementation, the network video recorder 112 is alsoconnected to a control system 120, which is typically housed in asecurity room of the building. However, the security room could also besecurity booth or located offsite, to list a few examples. In theillustrated example, the control system 120 communicates with an alertsystem 122, which generates alerts based on the metadata indicatingabandoned or removed objects. In further aspects, the alerts are basedon user parameters (e.g., such as during certain time periods) or basedon changes detected in specific regions within the field of viewcaptured by the security cameras 110.

In the illustrated example, the security system 100 is monitored and/orcontrolled by the security personnel 126 with a workstation 124. In theillustrated example, the security personnel 126 is a single securityguard.

There are multiple ways that the analysis system 115 can be deployedwithin the security system 100. Some examples of possible deploymentsare illustrated in FIGS. 1A, 1B, and 1C.

As illustrated in FIG. 1A, the analysis system 115 is implemented withinthe network video recorder 112. In this example, the video data from thesecurity cameras 110 are received over the network 104 at the videorecorder 112. The analysis system 115 is usually a process that runs onthe network video recorder 112 or a separate system implemented on aninterface to the network video recorder 112.

In a typical implementation a background model 117, which is generatedby the analysis system, is stored (non-permanently) in a processingpipeline of the analysis system 115. Additionally, the background model117 is continually updated as frames of the video data are processed bythe analysis system 115.

As illustrated in FIG. 1B, the analysis system 115 is part of theindividual security cameras 110 in another configuration. Here, theanalysis of video data is performed within the security cameras 110. Thevideo data and metadata are then transmitted over the network 104 to thenetwork video recorder 112, which then stores the video data andassociated metadata in the archive 114.

FIG. 1C illustrates yet another example. Here, the analysis system 115is a separate system that processes the video data stored in archive114. In this example, the video data captured by the security cameras110 are stored to the archive 114. Then, the analysis system 115accesses that video data from the archive 114, generates the associatedmetadata, and stores the metadata back into the metadata store 118 ofthe archive 114. The video data and metadata are typically correlated toeach other via timestamps and camera identifiers stored with themetadata. This enables the video data and metadata to also be correlatedwith the particular video camera 110 and event that gave rise to themetadata 118.

In some examples, all of the video data generated by the video cameras110 are stored in the video store 116 of the archive 114. In otherexamples, video data are only stored when the analysis system 115, uponanalyzing that video data, determines that an event of interest occurredand that the video should be stored.

FIGS. 1A-1C illustrate three different embodiments of the securitysystem 100. However, the security system could be embodied in many otherconfigurations and should not be limited to the illustrated examples.For example, in another embodiment of the security system 100, theanalysis system 115 is a network node that is different from the networkvideo recorder 112.

FIG. 2 is flowchart illustrating the steps performed by the analysissystem 115 of the security system 100 to analyze frames of video datacaptured by the security cameras 110.

In the first step 202, the background model is generated. In a typicalimplementation, the analysis system 115 analyzes the video data from thesecurity cameras 110 to generate the background model of the respectiveareas being monitored. The background model is the stationary portion ofa scene being monitored by each security camera 110 and is generallyonly updated slowly and is based on one or more frames of video data.

In the next step 204, the analysis system 115 receives a next frame ofvideo data from the security cameras 110. In step 206, the analysissystem 115 identifies foreground objects in the received frame. And, theanalysis system 115 matches foreground objects between successive framesof the video data in step 208. In the next step 210, foreground objectsare tracked to monitor how the foreground objects move and interactwithin the scene. In a typical implementation, the foreground objectsare identified with bounding boxes and unique identifiers and stored asmetadata as they move within the scene.

In the next step 212, the analysis system 115 determines if there areany changes to the background model. If there is no change to thebackground model, then the analysis system 115 continues to analyzeframes of video data and track foreground objects. If there are changesto the background model, then the analysis system 115 characterizes thechanges to the background model in step 214 such as whether objects havebeen abandoned in the scene or removed from the scene.

FIG. 3 is a flow chart illustrating the steps performed by the analysissystem 115 to characterize the objects as abandoned or removed accordingto a first embodiment.

In general, the analysis system 115 compares the background model to atleast one current frame by performing an image subtraction operation toidentify at least one patch of changed pixels (or changed pixelpatches). The image subtraction operation may identify a single changedpixel patch or a multiple patches of changed pixels. Next, the analysissystem 115 uses morphological image processing (e.g., erosion, dilation,open, close) to generate masks based on the changed pixel patches. Theerosion operation uses a structuring element to remove (i.e., erode)boundary regions in a group of pixels. The dilation operation uses thestructuring element to increase boundary regions in a group of pixels.Similar or different structuring elements may be used to perform theerosions and dilations. The open operation is an erosion operationfollowed by the dilation operation (using the same structuring element).The close operation is a dilation operation followed by the erosionoperation (using the same structuring element).

The analysis system 115 then applies the masks to the background modeland the current frames to determine whether the changed pixel patchesare characteristic of abandoned or removed objects.

In the first step 301, the analysis system 115 receives a current frameof the video data from one of the security cameras 110. In the next step302, the analysis system 115 performs an image subtraction operationbetween the current frame and background model 117 (for each pixel incurrent frame) to generate an inner mask (i.e., changed pixel patch).

The analysis system 115 then performs a dilation on the inner mask usinga 5×9 pixel pattern filter in step 304, which is also referred to as thestructuring element. The pixel pattern filter can vary in size andshape. For example, the pixel pattern filter could be square,rectangular, triangular, diamond, or circular, to list a few examples.The different shapes and sizes of the pixel pattern filter will affectthe result of the morphological operation applied to the image.

The analysis system 115 then performs an erosion on the dilated innermask using a 5×5 pixel pattern filter to generate an outer mask in step306. As in step 304, the pixel pattern filter applied on the dilatedinner mask can vary. Next, in step 308, an erosion is performed on theouter mask to generate a mask whose contour lies near the changed pixelpatches. This is referred to as an eroded outer mask.

In step 310, the analysis system 115 selects pixels within the stripbetween the outer mask and the eroded outer mask, which is referred toas a boundary mask. The contours of the inner mask(φ_(background model)) are dilated to create the dilated contours of theinner mask in step 311. The analysis system 115 then extracts edgepixels (e.g., Canny edge detection algorithm) within the boundary maskin the background model in step 312.

Gradient magnitudes of edge pixels in the boundary mask in thebackground model 117 are calculated in step 314, and in step 316, theanalysis system 115 sums a magnitude of gradients over all the edgepixels within the dilated contours of the inner mask(φ_(background model)) computed in step 311.

The analysis system 115 extracts edge pixels (e.g., Canny edge detectionalgorithm) within the boundary mask in the current frame in step 320.

In the next step 322, the analysis system 115 calculates gradientmagnitudes of edge pixels in the boundary mask in the current image. Instep 324, the analysis system 115 sums a magnitude of gradients over allthe edge pixels within the dilated contours of the inner mask(φ_(background model)) computed in step 311.

The analysis system 115 determines if φ_(background model) is greaterthan φ_(current image) by a threshold (δ). If φ_(background model) isgreater than φ_(current image)+δ, then the object is characterized asabandoned in step 330. If φ_(background model) is not greater thanφ_(current image)+δ, then the object is characterized as removed in step328.

FIG. 4 illustrates an example of how the inner mask 402 is modified withmorphological image processing.

The inner mask 402 is generated by the image subtraction operation(performed in step 302 in FIG. 3). In the next step (step 304), thecontour of the inner mask is dilated, which is referred to as thedilated inner mask contour 404, using a 5×9 pixel pattern filter. Thedilated inner mask contour 404 is then eroded with a 5×5 pixel patternfilter (step 306) to yield the outer mask 406. The outer mask 406 iseroded to yield the eroded outer mask 408 (step 308). The result is astrip of pixels 410, referred to as the boundary mask, between the outermask 406 and eroded outer mask 408.

FIG. 5 is an alternative embodiment of how the analysis system 115characterizes changes to the background model for step 214 andclassifies objects as abandoned or removed.

In the first step 602, the analysis system 115 receives a current frameof the video data from the security cameras 110. In the next step 604, adifference mask (e.g., inner mask) is generated to locate pixels thatdiffer between the received current frame and the background model 117.In a typical implementation, the difference mask is a binary image,which indicates pixels that differ between the current frame and thebackground model 117.

The analysis system 115 then computes a boundary mask from thedifference mask in step 606. In a preferred embodiment, the boundarymask is computed by locating pixels for which 3-6 elements are insidethe difference mask, when analyzing each of the pixels and its eightadjacent neighbor pixels. Thus, the boundary mask defines a contouraround the at least one patch of changed pixels. Alternatively, othermethods for computing the boundary mask may also be implemented.

In the next step 608, the analysis system 115 converts the differencemask to high precision representation (i.e., 8-bit) and blurs thedifference mask, for example by using a 7×7 windowed, normalizedGaussian kernel. Next, in step 610, the gradient of the blurreddifference mask is computed. The analysis system 115 then blurs thecurrent frame and the background model 117 to reduce noise andhigh-frequency variations, for example by using a 5×5 windowed,normalized Gaussian kernel in step 612.

Next in step 616, the analysis system 115 computes a similarity scorefor the current frame as the sum of the square of the dot productbetween the gradient vector computed for the blurred difference mask(i.e., step 610) and the gradient vector for the same pixel of thecurrent frame over all pixels in the boundary mask (computed in step606).

Next in step 618, the analysis system 115 computes a similarity scorefor the background model as the sum of the square of the dot productbetween the gradient vector computed for the blurred difference mask(i.e., step 610) and the gradient vector for the same pixel of thebackground model over all pixels in the boundary mask (computed in step606).

If the similarity score for the background model is higher than thecurrent frame, then the analysis system 115 characterizes the object asremoved in step 626. If the similarity score for the background model islower than the current frame, then the analysis system 115 characterizesthe object as abandoned in step 624.

FIGS. 6A-6C illustrate objects being abandoned or removed from a scene.In general, FIG. 6A illustrates a background model, FIG. 6B illustratesan object being abandoned, and FIG. 6C illustrates an object beingremoved.

In general, in the case of an abandoned object, the contour of thechanged pixel patch more closely resembles edges detected in the currentimage. On the other hand, in the case of a removed object, the contourof the changed pixel patch more closely resembles edges detected in thebackground model image.

FIG. 6A illustrates the background model 117. In the illustratedexample, the background model includes a first plant 506, a second plant508, a table 510, and a sofa 512.

FIG. 6B illustrates a frame 504 b of video data received from one of thesecurity camera 110. In this frame, a third plant 514 has been abandoned(added) to the background. The abandoned third plant 514 creates achanged pixel patch (i.e., area 516) around the region where the thirdplant 514 was abandoned. In this example, the contour of the changedpixel patch will more closely resemble edges detected in the currentimage.

FIG. 6C illustrates an alternative example of a frame 504 c of videodata received from one of the security cameras 110. In this frame, thesecond plant 508 has been removed. The removal of the second plant 508creates a changed pixel patch (i.e., area 518) where the plant waspreviously located. In this example, the contour of the changed pixelpatch will more closely resemble edges detected in the background modelimage.

While this invention has been particularly shown and described withreferences to preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the scope of the inventionencompassed by the appended claims.

What is claimed is:
 1. A method for analyzing video data in a securitysystem, the method comprising: comparing a background model to videodata to identify at least one patch of changed pixels between thebackground model and the video data; generating masks based on the atleast one patch of changed pixels that define contours around the atleast one patch of changed pixels; and applying the masks to thebackground model and the video data to determine whether the at leastone patch of changed pixels is characteristic of an abandoned object ora removed object.
 2. The method according to claim 1, further comprisingcomputing a similarity score for the video data as a sum of the squareof dot product between a gradient vector computed for a blurred mask anda gradient vector for the video data over border pixels of the patch ofchanged pixels.
 3. The method according to claim 1, further comprisingcomputing a similarity score for the background model as a sum of thesquare of dot product between a gradient vector computed for a blurredmask and a gradient vector for the background model over border pixelsof the patch of changed pixels.
 4. The method according to claim 1,further comprising associating metadata with the at least one frame andcreating alerts based on the metadata.
 5. The method according to claim1, further comprising performing morphological image processing on theat least one patch of changed pixels.
 6. The method according to claim5, wherein the morphological image processing includes dilations.
 7. Themethod according to claim 5, wherein the morphological image processingincludes erosions.
 8. The method according to claim 1, furthercomprising extracting edge pixels in the background model and the videodata with an edge detection algorithm.
 9. The method according to claim8, further comprising calculating gradient magnitudes of the extractededge pixels in the background model and the video data.
 10. The methodaccording to claim 9, further comprising summing gradient magnitudes ofall edge pixels in the background model and the video data.
 11. Themethod according to claim 1, further comprising blurring the video dataand background model to reduce noise and variations.
 12. The methodaccording to claim 1, further comprising converting the mask to a highprecision representation.
 13. The method according to claim 1, furthercomprising continually updating the background model based determinationof abandoned objects and/or removed objects.
 14. A method for analyzingvideo data in a security system, the method comprising: comparing abackground model to the video data to identify at least one patch ofchanged pixels between the background model and the video data; summinggradient magnitudes for edge pixels in the background model with respectto the at least one patch of changed pixels; summing gradient magnitudesfor edge pixels in the video data with respect to the at least one patchof changed pixels; and characterizing objects in the video data asabandoned or removed based on a comparison of the sums of gradientmagnitudes between the background model and the video data.
 15. Themethod according to claim 14, further comprising continually updatingthe background model based determination of the abandoned objects and/orthe removed objects.
 16. The method according to claim 15, wherein thebackground model is temporarily stored in a processing pipeline.
 17. Themethod according to claim 14, further comprising associating metadatawith the video data and creating alerts based on the metadata.
 18. Themethod according to claim 14, further comprising performingmorphological image processing on the at least one patch of changedpixels.
 19. The method according to claim 18, wherein the morphologicalimage processing includes dilations.
 20. A security system comprising:video cameras to capture video data; a video recorder to store thecaptured video data; an analysis system that analyzes the video data ofthe security system; and wherein the analysis system compares abackground model to the video data to identify at least one patch ofchanged pixels between the background model and the video data,generates masks based on the at least one patch of changed pixels, whichdefine contours around the at least one patch of changed pixels, appliesthe masks to the background model and the video data, and determineswhether the at least one patch of changed pixels is characteristic ofabandoned objects or removed objects.